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Abstract. We review the use of mean field theory for describing the 
dynamics of dense, randomly connected cortical circuits. For a simple 
network of excitatory and inhibitory leaky integrate-and-fire neurons, 
we can show how the firing irregularity, as measured by the Fano factor, 
increases with the strength of the synapses in the network and with the 
value to which the membrane potential is reset after a spike. Generalizing 
the model to include conductance-based synapses gives insight into the 
connection between the firing statistics and the high-conductance state 
observed experimentally in visual cortex. Finally, an extension of the 
model to describe an orientation hypercolumn provides understanding 
of how cortical interactions sharpen orientation tuning, in a way that is 
consistent with observed firing statistics. 



1 Introduction 

Neocortical circuits are highly connected: a typical neuron receives synaptic in- 
put from of the order of 10000 other neurons. This fact immediately suggests 
that mean field theory should be useful in describing cortical network dynamics. 
Furthermore, a good fraction, perhaps half, of the synaptic connections are local, 
from neurons not more than half a millimeter away, and on this length scale (i.e., 
within a "cortical column") the connectivity appears to be highly random, with 
a connection probability of the order of 10%. This requires a level of mean field 
theory a step beyond the kind used for uniform systems in condensed matter 
physics like ferromagnets. It has to describe correctly the fluctuations in the in- 
puts to a given network element as well as their mean values, as in spin glasses. 
The theory we use here is, in fact, adapted directly from that for spin glasses. 

A generic feature of mean field theory for spin glasses and other random sys- 
tems is that the " quenched disorder" in the connections (the connection strengths 
in the network do not vary in time) leads to an effectively noisy input to a single 
unit that one studies: Spatial disorder is converted to temporal. The presence of 
this noise offers a fundamental explanation for the strong irregularity of firing 
observed experimentally in cortical neurons. For high connectivity, the noise is 



Gaussian, and the correct solution of the problem requires its correlation func- 
tion to be found sclf-consistently. In this paper we summarize how to do this for 
some simple models for cortical networks. 

We focus particularly on the neuronal firing statistics. There is a long his- 
tory of experimental investigations of the apparent noisiness of cortical neurons 
ITWrUTTTrmTOTrTTTTW , but very little in the way of theoretical work 
based on network models. Our work begins to fill that gap in a natural way, since 
the full mean field theory of a random network is based on self-consistently calcu- 
lating the correlation function. In particular, we are able to identify the features 
of the neurons and synapses in the network that control the firing correlations. 

The basic ideas developed here were introduced in a short paper , and these 
models are treated in greater detail in several other papers j2l!'{l4l5ln] . Here we 
just want to give a quick overview of the mean field approach and what we can 
learn from it. 



2 Single column model, current-based synapses 

In all the work described here, our neurons are of the leaky integrate-and-fire 
kind, though it is straightforward to extend the method to other neuronal mod- 
els, based, for example, on Hodgkin-Huxley equations. In our simplest model, 
we consider networks of excitatory and inhibitory neurons, each of which re- 
ceives a synaptic connection from every other neuron with the same probability. 
Each such connection has a "strength," the amount by which a presynaptic spike 
changes the postsynaptic potential. In this model, these strengths are indepen- 
dent of the postsynaptic membrane potential ("current-based synapses"). All 
excitatory-to-excitatory connections that are present are taken to have the same 
strength, and analogously for the three other classes of connections (excitatory- 
to-inhibitory, etc.). However, the strengths for the different classes are not the 
same. In addition, excitatory and inhibitory neurons both receive excitation from 
an external population, representing "the rest of the brain" . (For primary sen- 
sory cortices, this excitation includes the sensory input from the thalamus.) This 
is probably the simplest generic model for a generic "cortical column" of spiking 
neurons. 

The network is taken to have A'^i excitatory and N2 inhibitory neurons. A 
given neuron (of either kind) receives synaptic input from every excitatory (resp. 
inhibitory) neuron with probability Ki/Ni (resp. K2/N2), with Ka/Na indepen- 
dent of a. In our calculations we take the connection density Ka/Na to be 10%, 
but the results are not very sensitive to its value as long as it is fairly small. 
Each nonzero synapse from a neuron in population b to one in population a is 
taken to have the value Jab/ V Kb- Synapses from the external population are 
treated in the same way, with strengths Jao/V^o- For simplicity, neurons in 
the external population are assumed to fire like stationary independent Poisson 
processes. We consider the limit Ka 00, Na 00, with Ka/Na fixed, where 
mean field theory is exact. 



The subthreshold dynamics of the membrane potential of neuron i in popu- 
lation a obey 

r\ii'^ 11°- ^ 

-^--^+m:Jt^s][ti (1) 

6=0 ] = 1 

where 5'j'(t) — 'Y^^ 5{t — t^^) is the spike train of neuron j in population b. The 
membrane time constant is taken to have the same value r for all neurons. We 
give the firing thresholds a narrow distribution of values (10% of the mean value, 
1). We take the firing thresholds 9a — I and the postfiring reset levels to be 0. 
We ignore transmission delays. 

The essential point of mean field theory is that for such a large, homoge- 
neously random network, as for an infinite-range spin glass |2UI21I22] . we can 
treat the net input to a neuron as a Gaussian random process. This reduces 
the network problem to a single-neuron one, with the feature that the statistics 
of the input have to be determined self-consistently from the firing statistics of 
the single neurons. This reduction was proved formally for a network of spiking 
neurons by Fulvi Mari |23 . 

Explicitly, the effective input current to neuron i in population a can be 
written 

int) = JablVK'bn + Bbxf + ^1-Kb/Nb^f{t)]. (2) 

b 

Here rt = Nf^^ J2j '^j is the average rate in population 6, 

xf^ is a unit-variance Gaussian random number, and £,i^{t) is a (zero- mean) 
Gaussian noise with correlation function equal to Cb{t — t'), the average corre- 
lation function in population b. For the contribution from a single population, 
labeled by 6, the first term in Q, which represents the mean input current, is 
larger than the other two, which represent fiuctuations, by a factor of 0{^/Kb): 
averaging over many independent input neurons reduces fluctuations relative to 
those in a single neuron by the square root of the number of terms in the sum. 
(For our way of scaling the synapse strengths, the factor y/Kb in the first term 
arises formally from adding Kb terms, each of which is proportional to l/^/JCb.) 

However, while the fiuctuation terms are small in comparison to the mean 
for a given input population b, small compared to the population-averaged input 
(the first term in (0)), we will see that when we sum over all populations the 
first term will vanish to leading order. What remains of it is only of the same 
order as the fiuctuations. Therefore fiuctuations can not be neglected. 

The fact that the fluctuation terms are Gaussian variables is just a conse- 
quence of the central limit theorem, since we consider the limit Kb — > oo. 

Note that one fiuctuation term is static and the other dynamic. The origin of 
the static one is the fact that the network is inhomogeneous, so different neurons 



will have different number of synapses and therefore different strengths of net 
time-averaged inputs. 

It is perhaps not immediately obvious, but the formal derivation f|23|. see 
also [21] for an analogous case) shows that the dynamic noise also originates 
from the random inhomogeneity in the network. It would be absent if there were 
no randomness in the connections, as, for example, in a model like ours but with 
full connectivity. The presence of the factor \/l — Ka/Na in the third term in ||2J) 
makes this point evident; in the general case the noise variance is proportional 
to the variance of the connection strengths. 



The mean field ansatz 

In any mean field theory, whether it is for a ferromagnet, a superconductor, elec- 
troweak interactions, or a neural network, one has to make an ansatz describing 
the state in question. This ansatz contains some parameters (generally called 
"order parameters"), the values of which are then determined self-consistently. 
Here, our "order parameters" are the mean rates r?,, their mean square values 
(which appear in Q, and the correlation functions Cb{t — t'). We make an ansatz 
for the correlation functions that describes an asynchronous irregular firing state: 
We take rf, to be time-independent and Cb{t — t') to have a delta- function peak 
(of strength equal to ri,) ai t — t' , plus a continuous part that falls off toward 
zero as \t — t'\ — > oo. We could also look, for example, for solutions in which 
Tb was time-dependent and/or Cb{t — t') had extra delta- function peaks (these 
might describe oscillating population activity ), but we have not done so. Thus, 
we cannot exclude the existence of such exotic states, but we can at least check 
whether our asynchronous, irregularly-firing states exist and are stable. 

We can find the mean rates, at least when they are low, independently of 
their fluctuations and the correlation functions: In an irregularly-firing state the 
membrane potential should fluctuate around a stationary value, with the noise 
occasionally and irregularly driving it up to threshold. In mean field theory, we 
have 

^ = -^+Ia{t), (4) 

where Ia{t) is given by Q. (We have dropped the neuron index i, since we are 
now doing a one-neuron problem.) From jSJ, we see that the leading terms in 
Ia{t) are large (cx ^/Kb), so if the membrane potential is to be stationary they 
must nearly cancel: 

2 

Y.Jab^/K'brb^Oil). (5) 

fc=0 

That is, the mean excitatory (6 = 0, 1) and inhibitory (6 = 2) currents must 
nearly balance. Therefore we call (O the balance condition. Defining Jab = 
Jab \J Kb I Kq , we can also write it in the form 

2 

Jabn = 0. (6) 

&=0 



The external rate tq is assumed known, so these two hnear equations can be 
solved for r;,, 6 = 1, 2. We can write the solution as 

2 ^ 

ra = -'^0 ]abJborQ, (7) 

6=1 

where by we mean the inverse of the 2x2 matrix with elements Jab, a,b = 
1,2. This result was obtained some time ago by Amit and Brunei |25I26I27) and, 
for a nonspiking neuron model, by van Vreeswijk and Sompolinsky |28I29| . 

However, a complete mean field theory involves the rate fluctuations within 
the populations and the correlation functions, and it is clear that if we want to 
understand something quantitative about the degree of irregularity of the neu- 
ronal firing, it is necessary to do the full theory. This cannot be done analytically, 
so we resort to numerical calculation. 

Numerical procedure 

Our method was inspired by the work of Eisfeller and Opper PU] on spin glasses. 
They, too, had a mean field problem that could not be solved analytically, so they 
solved numerically the single-spin problem to which mean field theory reduced 
their system. In our case, we have to solve numerically, the problem of a single 
neuron driven by Gaussian random noise, and the crucial part is to make the 
input noise statistics consistent with the output firing statistics. 

This requires an iterative procedure. We have to start with a guess about the 
mean rates, the rate fiuctuations, and the correlation functions for the neurons in 
the two populations. We then generate noise according to l(2Jl and simulate many 
trials of neurons driven by realizations of this noise. In these trials, the effective 
numbers of inputs Kb are varied randomly from trial to trial, with a Gaussian 
distribution of width \fKh, to capture the effects of the random connectivity in 
the network. We compute the firing statistics for these trials and use the result 
to improve our estimate of the input noise statistics. We then repeat the trials 
and iterate the loop until the input and output statistics agree. 

We can get a good initial estimate of the mean rates from the balance condi- 
tion equation Q, but this is harder to do for the rate fluctuations and correlation 
function. The method we have used is to do the initial trials with white noise in- 
put (of a strength determined by the mean rates). There seems to be no problem 
converging to a solution with self consistent rates, rate fluctuations and firing 
correlations from this starting point. 

More details of the procedure can be found in [2]. 

Some results 

As a measure of the firing irregularity, we consider the Fano factor F . It is 
defined as the ratio of the variance of the spike count to its average, where both 
statistics are computed over a large number of trials. It is easy to relate it to the 
correlation function, as follows. 




Fig. 1. Fano factor as a function of the overall synaptic scaling parameter Jg, 
for 3 values of the relative inhibition parameter g. 

If S{t) is a spike train as in m, the spike count in an interval from to T is 

i-T 

dtS{t). (8) 

"'0 

Its mean is just n — rdt — rT, and its variance is 

(n-n)2 = [ dt [ dt'{[Sit)-r){Sit')-r]) (9) 
Jo Jo 

The quantity in the averaging brackets in is just the correlation function. 
Changing integration variables from {t, t') to (t = ^(i + s = t — t') and taking 
T — > (X) gives 

F^l + - ( dsC{s). (10) 

For a Poisson process, C(s) = r(5(s), leading to F = 1. 

Thus, a Fano factor greater than 1 is not really "more irregular than a Pois- 
son process" , since any deviation of F from 1 comes from some kind of firing 
correlations. 

For this model we have studied how the magnitude of the synaptic strengths 
affects the Fano factor. We have used 



In Fig. n we plot F as a function of the overall scaling factor Js for three dif- 
ferent values of the relative inhibition strength g. Evidently, increasing synaptic 
strength in either way increases F . 

How can we understand this result? Let us think of the stochastic dynamics 
of the membrane potential u after a spike and reset, as described, for example, 
by a Fokker-Planck equation. Right after reset, the distribution of u is a delta- 
function at the reset level. Then it spreads out diffusively and its center drifts 
toward a quasi-equilibrium level. The speed of the spread and the width of the 
quasi-equilibrium distribution reached after a time t are both proportional to 
the synaptic strength. 

This distribution is only "quasi-equilibrium" because on the somewhat longer 
timescale of the typical interspike interval, significant weight will reach the ab- 
sorbing boundary at threshold. Nevertheless, we can regard it as nearly station- 
ary if the rate is much less than t^^. 

The center of the quasi-equilibrium distribution has to be at least a standard 
deviation or so below threshold if the neuron is going to fire at a low-to-moderate 
rate. Thus, since this width is proportional to the synaptic strengths, if we fix 
the reset at zero and the threshold at 1 the drift of the distribution after reset 
will be upward for sufficiently weak strengths and downward for strong enough 
ones. Hence, in the weak case, there is a reduced probability of spikes (relative 
to a Poisson process) for times shorter than t, leading to a refractory dip in 
the correlation function and a Fano factor bigger than 1. In the strong-synapse 
case, the rapid initial spread of the membrane potential distribution before it 
has a chance to drift very far downward leads to excess early spikes, a positive 
correlation function at short times, and a Fano factor bigger than 1. The relevant 
ratio is the width of the quasi-equilibrium membrane potential distribution (for 
this model, roughly speaking, J^) divided by the different between reset and 
threshold. 

The above argument applies even for neurons with white noise input. But 
in the mean field description the firing correlation induced by this effect lead to 
correlations in the input current, which amplify the effects. 

3 Model with conductance-based synapses 

In a second model, we add a touch of realism, replacing the current-based 
synapses by conductance-based ones. Then the postsynaptic potential change 
produced by a presynaptic spike is equal to a strength parameter multiplied by 
the difference between the postsynaptic membrane potential and the reversal 
potential for the class of synapse in question. In addition, we include a simple 
model for synaptic dynamics: we need no longer assume that the postsynaptic 
potential changes instantaneously in response to the postsynaptic spike. 
Now the subthreshold membrane potential dynamics become 




(12) 



6=0 i=l 



Here (^l is a nonspecific leakage conductance (taken in units of inverse time); it 
corresponds to r^^ in The Vb are the reversal potentials for the synapses 
from population b; they are above threshold for excitatory synapses and below 
for inhibitory ones, so the synaptic currents are positive (i.e., inward) and 
negative (outward), respectively, in these cases. The time-dependent synaptic 
conductances 5^"'f,(i) reflect the firing of presynaptic neuron j in population &, 
filtered at its synapse to postsynaptic neuron i in population a: 

3:^(t) = ^ f dt'K{t~t')S,{t') (13) 

V^f) J -oo 

when a connection between these neurons is present; otherwise it is zero. (We 
assume the same random connectivity distribution as in the previous model.) 
We have taken the synaptic filter kernel K{t) to have the simple form 

m = , (14) 

representing an average temporal conductance profile following a presynaptic 
spike, with characteristic opening and closing times ri and T2. This kernel is 
normalized so that / (ltK{t) — 1; thus, the total time integral of the conduc- 
tance over the period after an isolated spike is equal to Hence, for 
very short synaptic filtering times, this model looks like Q with a membrane 
potential-dependent Jf^ equal to g^b{'^\{t) — Vf,)/V^- We take the (dimension- 
less) parameters 5^^, like the Jab in the previous model, to be of order 1, so we 
anticipate a large {0{'\/ Kb)) mean current input from each population b and, 
in the asynchronously-firing steady state, a near cancellation of these separately 
large currents. 

In mean field theory, we have the effective single-neuron equation of motion 

= -gLUa{i) -Y^CJamUait) ~ H), (15) 
fc=0 

in which the total effect of population b on a neuron in population a is a time- 
dependent conductance gab{t) consisting of a population mean 



(gab) = \/Kbglbn, (16) 

static noise of variance 

and dynamic noise with correlation function 

{5gab{t) 5gabit')) = (l - ^) {9lb)^Cb{t ~ t'), (18) 



where 



Cb{t-t')= dtiK{t-h) dt2K{t' -t2)Cb{tiM) (19) 



is the correlation function of the synaptically fihered spike trains of population 
h. 



The balance condition 

As for the model with current-based synapses, we can argue that in an irregu- 
larly, asynchronously-firing state the average du"/dt should vanish. From H15|l 
we obtain 

QLU^ + Y,9lb^brb{u^ - Vi) = 0. (20) 

b 

Again, for large connectivity the leakage term can be ignored. In contrast to 
what we found in the current-based case, now the balance condition requires 
knowing the mean membrane potential u^. However, we will see that in the 
mean field limit the membrane potential has a narrow distribution centered just 
below threshold. Since the fluctuations are very small, the factor Ua — Vb ~ Oa — Vb 
in (|12|l can be regarded as constant, and we are effectively back to the current- 
based model. Thus, defining 

Jit = glbiVb - ^a), (21) 

we can just apply the analysis from the current-based case. 
High-conductance state 

It is useful to measure the membrane potential relative to u^. So, writing Ua = 
5ua(t) and using the balance condition (|20|l . we find 

-9tot{t)5ua + ^ SgabmV, - u^), (22) 



where 



At 

b 

gtot{t)^gL+Y,9ab{t) (23) 

b 

= 9L+Y.i^b9lbrb + 59ab{t% (24) 

b 

with Sgab{t) the fluctuating parts of gab{t), the statistics of which are given 
by (|17|l and (|18|) . This looks like a simple leaky integrator with current input 
'^iiSgab{t)iVb — u^) and a time-dependent effective membrane time constant 
equal to 5tot(^)^^- Following Shelley et al. TP, H22I) can be further rearranged 
into the form 

^^-gtotmSua-SV^it)], (25) 




Fig. 2. The membrane potential u{t) follows the effective reversal potential Vs{t) 
closely, except when Vs is above threshold. Here, the threshold is 1 and the reset 
0.94. 



with the "instantaneous reversal potential" (here measured relative to Ua) given 
by 

Eq. H25|) says that at any instant of time, Sua is approaching SV^{t) at a rate 
gtot{t). 

For large Kb, gtot is large {0{y/Kb)), so the effective membrane time constant 
is very small and the membrane potential follows the fluctuating SV^{t) very 
closely. Fig. [5] shows an example from one of our simulations. This is the main 
qualitative difference between mean field theory for the model with current- 
based synapses and the one with conductance-based ones. It is also the reason 
we introduced synaptic filtering into the present model. In the current-based 
one, the membrane potential filtered the input current with a time constant r 
which we could assume to be long compared with synaptic time constants, so 
we could safely ignore the latter. But here the effective membrane time constant 
becomes shorter than the synaptic filtering times, so we have to retain the kernel 
K{t) (|14|l . Here we have argued this solely from the fact that we are dealing with 
the mean field limit, but Shelley et al. argue that it actually applies to primary 
visual cortex (see also [5^1. 

We also observe that in the mean field limit, both the leakage conductance 
and the fluctuation term 5gab{t) are small in comparision with the mean, so we 



can approximate gtot(t) by a constant: 



fftot = y^bgW (27) 

b 

Furthermore, the fluctuations 6V^{t) in the instantaneous reversal potential H26|l 
are then of order l/\/Ki,'. membrane potential fluctuations can not go far from 
u^. But Vi^{t) must go above threshold frequently enough to produce firing 
at the self-consistent rates. Thus, must lie just a little below threshold, as 
promised above. Hence, at fixed firing rates, the conductance-based problem 
effectively reduces to a current-based one with a very small effective membrane 
time constant Tcff = g^^l and synaptic coupling parameters given by (|21(l . 
Of course, as we increase the firing rates of the external population and thereby 
increase the rates in the network, we will change g^ot, making both Toff and the 
fiuctuations SV^(t) correspondingly smaller. 

If we neglected synaptic filtering, the resulting dynamics would be rather 
trivial. It would be self-consistent to take the input current as essentially white 
noise, for then excursions of SV^ (t) above threshold would be be uncorrelated, 
and, since the membrane potential could react instantaneously to follow it up to 
threshold, so would the firing be. (Simulations confirm this argument.) 

Therefore, the synaptic filtering is essential. It imparts a positive correla- 
tion time to the fluctuations SV^{t), so if it rises above threshold it can stay 
there for a while. During this time, the neuron will fire repeatedly, leading to 
a positive tail in the correlation function for times of the order of the synaptic 
time constants. This broader the kernel K(t — t'), the stronger this effect. In the 
self-consistent description, this effect feeds back on itself: SV^{t) acquires even 
longer correlations, and these lead to even stronger bursty firing correlations. 

Thus, the mean field limit Ki, — > oo can be pathological in the conductance- 
based model with synaptic filtering. However, here we take the view that mean 
field theoretical calculations may still give a useful description of real cortical 
dynamics, despite that fact that real cortex is not described by the Kb — > oo 
limit. For example, the true effective membrane time constant is not zero, but, 
according to experiment [32], it is significantly reduced from its in vitro value 
by synaptic input, probably to a value less than characteristic synaptic 
filtering times. Doing mean field theory with moderately, but not extremely 
large connectivities can describe such a state in a natural and transparent way. 

As in the current-based model, the Fano factor grows with the ratio of the 
membrane potential distribution to the threshold-reset difference. It also grows 
with increasing synaptic filtering time, as argued above. Fig. O shows F plotted 
as a function of T2, with ri fixed at 1 ms, for a set of reset values. 

4 Orientation hypercolumn model 

Finally, we try to take a step beyond homogeneously random models to describe 
networks with systematic structure in their connections. We consider the ex- 
ample of a hypercolumn in primary visual cortex: a collection of n orientation 



14 



Reset 


= 0.96 


Reset 


= 0.9 


Reset 


= 0.8 



10 




3 



Fig. 3. Fano factor as a function of T2 for 3 reset values. 



columns, within which each neuron responds most strongly to a stimulus of a par- 
ticular orientation. The hypercolumn contains columns that cover the full range 
of possible stimulus orientations from to tt. It is known that columns with simi- 
lar orientation selectivities interact more strongly than those with dissimilar ones 
(because they tend to lie closer to each other on the cortical surface). We build 
this structure into an extended version of our model, which can be treated with 
essentially the same mean field methods as the simpler, homogeneously random 
one. In the version we present here, we revert to current-based synapses, but it 
is straightforward to construct a corresponding model with conductance-based 
ones. 

A study of a similar model, for non-spiking neurons, was reported by Wolf et 
al. . Those authors have also simulated a network of spiking neurons like the 
one described here |34| . complementing the mean-field analysis we give here. 

Each neuron now acquires an extra index 9 labeling the stimulus orientation 
to which it responds most strongly, so the equations of motion for the membrane 
potentials become 

^-~-^ + it{eM + Y.Y.T.Jtr sf{t). (28) 

6=1 e'=i j=i 

The term I^{9,9o,t) represents the external input current for a stimulus with 
orientation 9q. (In this section we set all thresholds equal to 1, and 6 refers 
only to orientation.) We assume it comes through diluted connections from a 



population of Nq ^ 1 Poisson neurons which fire at a constant rate tq: 



/r(0,0o,t) = E4°(^'^o)^°W- (29) 

j 



As in the single-column model, we take the nonzero connections to have the 
value 



J5°(0,^?o) = ^ (30) 



but now we take the connection probability to depend on the difference between 
9 and 9o, according to 

Poi0,9o)^^[l + ecos2{9-9o)]. (31) 

This tuning is assumed to come from a Hubel-Wiesel feed-forward connectivity 
mechanism. The general form has to be periodic with period tt and so would have 
terms proportional to cos 2m{9 — 9^) for all to, but here, following Ben-Yishai et 
al. 1^ , we use the simplest form possible. We have also assumed that the degree 
of tuning, measured by the anisotropy parameter e, is the same for inhibitory 
and excitatory postsynaptic neurons. 

Assuming isotropy, we are free to measure all orientations relative to Sq, so 
we set 9q — Q from now on. 

Similarly, we take the nonzero intracortical interactions J^^'^^ to be 

ja9,b9' Jab 

- ^ ^ 

and take the probability of connection to be 

Ki 

Pabi9 - 9') ^-^[1 + J cos2{9- 9% (33) 

Analogously to Pah is independent of both the population indices a and b, 
since we always take Kb/Ni, independent of b. 

In real cortex, cells are arranged so they generally lie closer to ones of similar 
than to ones of dissimilar orientation preference, and they are more likely to have 
connections with nearby cells than with distant ones. This is the anatomy that 
we model with . In ^ and [S| a slightly different model was used, in which the 
connection probability was taken constant but the strength of the connections 
was varied like The equations below for the balance condition and the 

column population rates are the same for both models, but the fluctuations are 
a little different. 

As for the input tuning, the form (|33() is just the first two terms in a Fourier 
series, but again we use the simplest possible form for simplicity. 



Balance condition and solving for population rates 



The balance condition for the hypercolumn model is simply that the net synaptic 
current should vanish for each column 6. Going over to a continuum notation by 
writing the sum on columns as an integral, we get 

/7r/2 1^/ 
— Jab[l + icos2{e-e')]^brbi9')^0. 

(34) 

We have to distinguish two cases: broad and narrow tuning. In the broad case, the 
rates rb{6) are positive for all 6. In the narrowly-tuned case (the physiologically 
realistic one), ri,{0) is zero for \6\ greater than some 6c, which we call the tuning 
width. (In general 9c could be different for excitatory and inhibitory neurons, 
but with our a- independent e in (|31|l and a- and 6- independent 7 in H33|) , it turns 
out not to.) 

In the broad case the integral over 6' can be done trivially with the help of 
the trigonometric identity cos (A — B) = cos A cos B + sin A sin B and expanding 
?'f)(^') = fbfi + f'b.2 cos26'' + • • •• We find that the higher Fourier components 
Tb,2m, TO > 1, do not enter the result: 

^/K'oJao{l + ecos2e)ro + ^ ^/K'bJab{rbfl + \irb,2COs2e) = 0. (35) 

b 

If (|35|) is to hold for every 6, the constant piece and the part proportional to 
cos 26 both have to vanish: for each Fourier component we have an equation like 
Thus we get a pair of equations like (TJl: 

^afi = -y^\y\bJbOra ra,2 = V'[J"^]afc^60?'0 (= — ?'a,o), (36) 



where Jab = Jaby/Kb/Ko, as in the simple model. 

This solution is acceptable only if e < 7/2, since otherwise ra{6) will be 

negative for 6 > 5 cos~l(-^a,o/7'a,2)■ 
Therefore, for e > 7/2, we make the ansatz 

r a{9) ^raa{cos2e - cos26c) (37) 

(i.e., we write Vafi as — ra_2 cos 26'c) for \9\ < 6c and ra{6) = for 16*1 > 9c. We 
put this into the balance condition H34|l . Now the integrals run from —9c to 9c, 
so they are as trivial as in the broadly-tuned case, but the ansatz works and we 
find 



-'^aO^'O + ^ Jabrb,2fo{9c) = 0, eJaO^'O + 7 X! Jabrb,2f2{6c) = 0, (38) 

where 



b b 



h{6c) = -(sin 20, - 20, cos 20,), /2(0,) = i(0, - i sin 40,), (39) 
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(The algebra here is essentially the same as that in a different kind of model 
studied by Ben-Yishai et al. [321; see also 

Eqns. H38|) can be solved for 9c and ra,2, a — 1,2. Dividing one equation by 
the other leads to the following equation for Oc- 

MA = 1 (40) 

Then one can use either of the pairs of equations (|38() to find the remaining 
unknowns fa, 2: 

ra,2 = J2^r\bJboro- (41) 

The function /2(^c)//o(^c) takes the value 1 at = and falls monotonically 
to \ at 9c = it/2. Thus, a solution can be found for ^ < e/7 < 1. For e/7 i, 
7''/2 and we go back to the broad solution. For e/7 — > 1, ^ 0: the tuning 
of the rates becomes infinitely narrow. Note that stronger tuning of the cortical 
interactions (bigger 7) leads to broader orientation tuning of the cortical rates. 
This possibly surprising result can be understood if one remembers that the 
cortical interactions (which arc essentially inhibitory) act divisively (see, e.g., 
^ and gU). 

Another feature of the solution is that, from (|40|l . the tuning width does 
not depend on the input rate tq, which we identify with the contrast of the 
stimulus. Thus, in the narrowly-tuned case, the population rates in this model 
automatically exhibit contrast-invariant tuning, in agreement with experimental 
findings [HZI- We can see that this result is a direct consequence of the balance 
condition. 

However, we should note that individual neurons in a column will exhibit 
fluctuations around the mean tuning curves which are not negligible, even in 
the mean- field limit. These come from the static part of the fluctuations in the 
input current (like the second term in |(2Jl for the single-column model), which 
originate from the random inhomogeneity of the connectivity in the network. 

As for the single-column model, the full solution, including the determination 
of the rate fluctuations and correlation functions, has to be done numerically. 
This only needs a straightforward extension of the iterative procedure described 
above for the simple model. 



Tuning of input noise 

We now consider the tuning of the dynamic input noise. Using the continuum 
notation, we get input and recurrent contributions adding up to 

{6Ia{9,t)6Ia{e,t'))^Jl^{\ + ecos2e)ro5{t-t') 

+ E r —Jlb[^^icos2{e~e')]c,{e\t-t'), (42) 



where Cb{0' , t — t') is the correlation function for population h in column 9' . We 
can not proceed further analytically for t ^ t' , since this correlation function has 
to be determined numerically. But we know that for an irregularly firing state 
Ch{0, t — t') always has a piece proportional to ri,{d)S{t — t'). This, together with 
the external input noise, gives a flat contribution to the noise spectrum of 

hm {\5Ia{9,uj)\^) = j2o(l + ecos2%o+V / —J!,[l + ^cos2{e-0')]n{e'). 

(43) 

The integrals on Q' are of the same kind we did in the calculation of the rates 
above, so we get 

hm (|<54 (0, t^) n = ( 1 + e cos 2^?)ro + V J.^r^.a [/o (Qc )+lWc) cos 20] 

h 

= ro(l + ecos20)(j2„ - ^ jf,[J-i],, J,o), (44) 

he 

where we have used (|40f) and l|41|l to obtain the last line. Thus, the recurrent 
synaptic noise has the same orientation tuning as that from the external input, 
unlike the population firing rates, which are narrowed by the cortical interac- 
tions. 

Output noise: tuning of the Fano factor 

To study the tuning of the noise in the neuronal firing, we have to carry out the 
full numerical mean field computation. Fig. 0] shows results for the tuning of the 
Fano factor with 6*, for three values of the overall synaptic strength factor J^. 
For small Js there is a minimum in F at the optimal orientation (0), while for 
large Jg there is a maximum. It seems that for any J^, is either less than 1 
at all angles or greater than 1 at all angles; we have not found any cases where 
the firing changes from subpoissonian to superpoissonian as the orientation is 
varied. 

5 Discussion 

These examples show the power of mean field theory in studying the dynamics of 
dense, randomly-connected cortical circuits, in particular, their firing statistics, 
described by the autocorrelation function and quantities derived from it, such 
as the Fano factor. One should be careful to distinguish this kind of mean field 
theory from ones based on "rate models" , where a function giving the firing 
rate as a function of the input current is given by hand as part of the model. 
By construction, those models can not say anything about firing statistics. Here 
we are working at a more microscopic level, and both the equations for the 
firing rates and the firing statistics emerge from a self-consistent calculation. We 
think it is important to do a calculation that can tell us something about firing 
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Fig. 4. Fano factor as a function of orientation 6 for 3 values of Js- 



statistics and correlations, since the irregular firing of cortical neurons is a basic 
experimental fact that should be explained, preferably quantitatively, not just 
assumed. 

We were able to see in the simplest model described here how this irregularity 
emerges in mean field theory, provided cortical inhibition is strong enough. This 
confirms results of |25I26I27] . but extends the description to a fully self-consistent 
one, including correlations. 

It became apparent how the strength of synapses and the post-spike reset 
level controlled the gross characteristics of the firing statistics, as measured by 
the Fano factor. A high reset level and/or strong synapses result in an enhanced 
probability of a spike immediately after reset, leading to a tendency toward 
bursting. Low reset and/or weak synapses have the opposite effect. 

Visual cortical neurons seem to show typical Fano factors generally some- 
what above the Poisson value of 1. They have also been shown to have very 
high synaptic conductances under visual stimulation. Our mean-field analysis 
of the model with conductance-based synapses shows how these two observed 
properties may be connected. 

In view of the large variation of Fano factors that there could be, it is perhaps 
remarkable that observed values do not vary more than they do. We like to 
speculate about this as a coding issue: Any constrained firing correlations imply 
reduced freedom to encode input variations, so information transmission capacity 
is maximized when correlations are minimized. Thus, plausibly, evolutionary 
pressure can be expected to keep synaptic strengths in the right range. 



Finally, extending the description from a single "cortical column" to an array 
or orientation columns forming a hypercolumn provided a way of understanding 
the intracortical contribution to orientation tuning, consistent with the basic 
constraints of dominant inhibition, irregular firing and high synaptic conduc- 
tance. 

These issues can also be addressed directly with large-scale simulations, as 
in However mean field theory can give some clearer insight (into the re- 
sults of such simulations, as well as of experiments), since it reduces the network 
problem to a single-neuron one, which we have a better chance of understand- 
ing. So we think it is worth doing mean field theory even when it becomes as 
computationally demanding as direct network simulation (as it does in the case 
of the orientation hypercolumn j34| ) . At the least, comparison between the two 
approaches can allow one to identify clearly any true correlation effects, which 
are, by definition, not present in mean field theory. 

Much more can be done with mean field theory than we have described here. 
First, as mentioned above, it can be done for any kind of neuron, even a multi- 
compartment one, not just a point integrate-and-fire one. At the level of the 
connectivity model, the simple model described by H33|l can also be extended 
to include more details of known functional architecture (orientation pinwheels, 
layered structure, etc.). It is also fairly straightforward to add synaptic dynamics 
(not just the average description of opening and closing of the channels on the 
postsynaptic side described by the kernel K{t — t') in (231). One just has to add 
a synaptic model which takes the spikes produced by our single effective neuron 
as input to a synaptic model. Thus, the path toward including more potentially 
relevant biological detail is open, at non-prohibitive computational cost. 
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